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THE  ANALYSIS  OF  RESIDUALS 


1.   Introduction 


1.1  General  Comments. 

An  analysis  of  the  residuals — the  observed  value  minus  the  fitted  value- 
can  answer  many  useful  and  sometimes  necessary  questions  concerning  data 
analysis.   One  of  the  most  important  of  these  questions  concerns  the  assump- 
tions Inherent  in  a  classical  least-squares  analysis.  This  will  be  the 
central  theme  of  this  paper. 

The  assumptions  underlying  a  classical  least-squares  analysis  (and  the 
resulting  F-test)  are  very  reasonable  for  most  situations.  Also,  the  F-test 
has  been  found  to  be  robust.   However,  in  some  situations  one  may  doubt  the 
validity  of  these  assumptions.   If  large  amounts  of  data  are  available  or 
long-term  projects  are  contemplated,  it  seems  reasonable  to  first  test  these 
assumptions  before  doing  the  calculations  and  tests  of  h3rpothe8is.   This  may 
lead  to  improvements  in  the  consequent  analysis,  such  as  use  of  nonparametric 
techniques  or  transformations. 

This  paper  will  present  both  graphic  methods  and  test  statistics  for 
testing  assumptions.   The  distributions  of  test  statistics  will  be  found 
under  the  null  hypothesis.  Methods  of  modifying  residuals  and  some  comments 
on  outliers  will  be  presented.  An  example  will  be  given  to  Illustrate  the 
procedures.  The  paper  will  conclude  with  some  stated  limitations  and 
cautions. 


1.2  Defining  the  Problem  in  Mathematical  Notation. 

Let  Y  be  an  (nxl)  vector  of  given  observations  which  are  claimed  to  be 
independent  determinations  of  the  (nxl)  vector  of  means  V   such  that 

(1.2a)  ii-Al   , 

vhere  A  is  a  (nxp)  matrix  of  given  coefficients  and  jj  is  a  (pxl)  vector  of 
unknovm  parameters.  Let  ^  denote  the  estimates  of  Q_  obtained  by  least- 
squares.  Then  the  fitted  values  become  an  (nxl)  vector  X  ^uch  that 

(1.2b)  I  -  A  8   , 

and  the  residuals  become  an  (nxl)  vector  Z   such  that 

(1.2c)  Z  -  I  -  I  •  ' 

Let  £  «  (q  .)  be  an  iderapotent  positive-semidefinite  symmetric  matrix 
which  takes  (y.)  into  (z . ) ;  i.e., 


(1.2d) 


'i"j^Vj   • 


If  A  has  rank  (n-v) ,  then  Q_  has  rank  v.  To  show  this,  note  that  the 

linear  subspace  C  generated  by  the  rows  (columns)  of  A  has  dimension  (n-v). 

Also,  Z   is  the  projection  of  all  n-component  vectors  orthogonal  to  C — call 

it  C.   C  has  dimension  v  which  means  ^  has  rank  v  (see  Rao,  1961).   In  an 

analysis  of  variance,  v  becomes  the  error  degrees  of  freedom. 

Given  A,   9  ,6  ,...,8     is  chosen  so  that  the  columns  of  A  are 
— '   12      ^"'^  ~ 

linearly  independent. 


From  (1.2a)  and  (1.2b), 

Y  -  Y  -  a  Y   . 

A  A  4 

It  can  be  shovm  (Graybill,  1961)  that  Y  -  A  8.  -  A  (A'A)"  A'Y.  Let 
V  -  (A'A)"^.  Then, 

Y  -  A  V  A'Y  '  Oil     , 

S  -  i  -  A  V  A'   . 

The  trace  of  (q^^)  "  v. 

In  developing  certain  statistics  It  Is  convenient  and  sometimes 
necessary  to  put  restrictions  on  A  and  ^.  They  are  the  following: 


(1.2e)       (1)  C  contains  the  unit  vector;  that  is,  one 

of  the  parameter  set  0_  is  the  general  mean, 
so  that  the  corresponding  column  of  A 
consists  entirely  of  one's. 


(1.2f)        (2)  The  diagonal  elements  of  Q_  are  all  equal. 

A  consequence  of  (1.2e)  is  that  each  row  or  column  of  ^  sum  to  zero.  A 
consequence  of  (1.2f)  is  that  each  diagonal  element  of  ^  is  equal  to  v/n» 
since  the  sum  of  the  diagonal  elements  of  a  symmetric  idempotent  matrix  Is 
equal  to  its  rank. 

Elsenhart  (1947)  discusses  the  assumptions  necessary  for  the  analysis 
of  variance,  but  for  the  purposes  of  this  paper  it  will  be  stated  that  the 
estimation  of  6^  by  least-squares  procedures  can  be  shown  to  be  satisfactory 
If  the  following  three  assumptions  are  met: 


(1.2g) 
(1.2h) 


(1)  The  effects  in  the  model  are  additive. 


(2)  The  error  variance  is  independent  of 
the  mean  (the  components  of  Y  are 
realizations  of  independent  random 
variables  all  with  equal  variances). 


(1.21) 


(3)  The  components  of  Y  are  realizations 
of  independent  random  variables  all 
normally  distributed.  \ 


Since  the  emphasis  in  this  paper  will  be  placed  on  the  two-way  classifi- 
cation, the  following  notation  will  be  used  in  certain  situations. 

The  n  -  re  observations  y,  ,  1  -  1,2 r,  and  J  ■  1,2,. ..,c,  will 

have  row  means  y.  "  z,Y.  /c,  column  means,  y  .  -  J  Y.  ./r,  and  a  grand  mean 

y  -  Jy  /n.  The  fitted  values  will  be  denoted  by 


ij 


ij 


^ij  ■  ^i.  •"  y.j  -  y  • 


and  the  residuals  will  be  denoted  by 


'ij  ■  "ij  "  ^ij   • 


The  analysis  of  variance  table  will  be  as  follows: 


Source 

dF 

SS 

Rows 

r-1 

SS-R  -  cj(y 
i  ^• 

-y)^ 

Columns 

c-1 

SS-C  -  r^(y 

-y)2 

Residual 

V 

In  order  to  get  precise  tests  of  the  assumptions  (1.2g)  -  (1.2i),  one 
would  need  large  samples  and  several  sets  of  data.   This,  however,  should 
not  keep  one  from  examining  these  assumptions  on  small  samples  or  a  single 
data  set,  but  rather  should  indicate  the  weight  put  on  the  results  of  such 
an  examination.   It  should  be  noted  that  some  of  the  procedures  discussed  In 
this  paper  require  modification  when  used  for  combined  data  sets.  The  main 
consideration  in  combining  the  data  sets  is  the  equality  of  variances.  This 
usually  can  be  corrected  by  using  standardized  residuals. 

2.   Graphic  Techniques 

Probably  the  most  revealing  and  most  important  of  all  statistical 
techniques  is  the  graph.  This  section  presents  some  graphic  techniques, 
using  residuals,  for  testing  assumptions  (1.2g)  -  (1.21). 

2.1  First  Scatter  Plot. 

A  good  way  to  begin  an  examination  of  residuals  Is  to  plot  them  against 
the  fitted  values.   With  increased  use  of  computers,  most  authors  suggest 
routine  calculation  of  the  residuals  on  each  analysis.   The  suggested  scatter 
plot  is  a  simple  next  step.  Outliers  will  show  up  as  Isolated  points.  As 
a  guide  for  Interpretation  of  this  plot,  note  the  following: 

(1)  The  expected  value  of  the  residuals  is  zero, 

(2)  The  linear  regression  of  the  residuals  on 
the  fitted  values  has  zer<2  slope. 

A  dependence  of  variability  on  level  of  response  will  reveal  itself  by 
causing  greater  dispersion  in  some  sections  of  the  scatter  plot  than  in 
others.  A  non-linear  graph  will  Indicate  that  the  effects  in  the  model  are 


not  additive.  The  technique  may  be  used  on  a  combination  of  several  data 
sets  if  the  residuals  are  standardized. 

2.2  FUNOP. 

FUNOP  (Full  Normal  Plot)  is  a  graphic  technique  suggested  by  Tukey  (1962) 
and  Anscombe  and  Tukey  (1963),  which  has  greater  sensitivity  than  the  scatter 

plot  or  the  curomulative  plot  on  normal  probability  paper. 

t 
Let  z  represent  the  median  of  the  residuals  and  a.  .  be  a  value 

chosen  to  be  typical  of  the  k-th  value  from  the  bottom  in  a  sample  of  n 

from  a  unit  normal  distribution.  Many  choices  are  available  for  \/J^• 

This  paper  will  use  Table  XX  of  Fisher  and  Yates  (1963).  A  plot  of 

(z.  -  2)/a,  y   against  k  (omitting  the  middle  third  of  the  k's)  has  the 

following  properties: 

(1)  A  single  outlier  will  be  revealed  by  a 
large  value  for  k  -  1  or  k  "  n. 

(2)  If  the  plot  turns  up  at  both  ends,  one 
should  suspect  that  he  is  either  dealing 
with  a  long-tailed  distribution  or  a 
number  of  outliers  of  each  sign. 

(3)  Skewness  will  be  revealed  by  the  plot 
being  higher  at  one  end  than  at  the 
other. 

Therefore,  a  straight  line  with  zero  slope  would  indicate  that  assumptions 
(1.2g)  -  (1.21)  are  met. 

2.3  Second  Scatter  Plot. 

This  technique  is  presented  by  Anscombe  and  Tukey  (1963)  as  a  method 
of  detecting  removable  nonaddltlvlty.  Consider  the  regression  coefficient 
f  of  the  residuals  regressed  on  the  fitted  values.  It  can  be  shown  (see 
Section  3.1)  that  f  -  B/A,  where 


B  -  h^y^ 


2 


and  A  is  the  residual  sum  of  squares  obtained  by  doing  the  conventional 
analysis  of  variance  on  the  square  of  the  fitted  values.   If  one  plots  the 
new  residuals  (Z  )  against  the  old  residuals  (Zj^  J »  removable  nonaddltivity 
will  show  up  as  a  distinct  straight  line.  A  test  of  significance  for 
nonaddltivity  using  B  and  A  is  given  in  Section  3.1. 

3.   Test  Statistics 

Test  statistics  presented  in  this  section  are  not  claimed  to  have 
sweeping  statistical  properties.  They  are,  however,  claimed  to  be  excellent 
guides,  in  absence  of  better  techniques,  for  handling  the  Important  problem 
of  testing  the  assumptions  necessary  for  a  classical  least-squares  analysis 
((1.2g)  -  (1.2i)). 

3.1  Additivity. 

Tukey  (1947)  stated  that  failure  of  the  asstmption  of  additivity  (1.2g) 
may  be  more  serious  than  the  others  for  tests  of  significance  in  an  analysis 
of  variance.   In  order  to  test  additivity,  consider  the  following  statistic 
due  to  Anscombe  (1961) :  ' 

Ivl 

(3.1a)  f  - 


,^/lJ^i'^J 


2 


where  (z  )  and  (q. .)  are  as  defined  in  Section  1.2  and  Y^  is  a  matrix  of  the 

square  of  the  fitted  values  (see  1.2b).   It  is  claimed  that  f  is  a  rough 

estimate  of  *  in  some  model  y-x+^(x'-y)^,  where  y  is  some 

o  o 


convenient  central  value.  Note  also  that  f  -  B/A,  as  discussed  in  Section 
2.2. 

The  f  statistic  is  motivated  by  the  fact  that 


E(z^)  -  E 


l]Vi, 


or 


E(z^)  -  E 


jqij(xj  +  *(xj-y,)2)  . 


If  the  X  are  distributed  as  N(p  ,0^)  and  (1.2d)  holds,  then 


Next,  note  that  restriction  (1.2e)  implies  that 


(3.1b) 


[q.   .  (constant)  ■  0   , 


since  any  row  or  column  of  (q^j)  sums  to  zero.  Also,  (1.2e)  implies  that 


(3.1c) 


Iq^jUj-O   . 


since 


2  iL  -  (i  -  A(A'A)"-^A')A  1-0   , 


From  (3.1c), 


By  adding  and  subtracting  (p.  -p  )^  and  combining  terms,  it  follows  that 
(3.1d)    E(z^)  -  ♦Iq^jElxjZ  -  2y^(Xj  -  p^)  -  v^^   +  (Uj  -  U^)^)   . 


Since  E(x  )  ■  W^. 

(3.1e)  E(-2y^(Xj  -  y^))  -  0  . 

If  X.  is  N(y  ,0^),  then 

,1 
(3. If)  e(x  2  -  p_j2j  .  (,2   . 

From  (3.1c), 

(3.1g)  ^hi.o^   -  0  • 

J  ^ 

By  substituting  (3.1e),  (3. If),  and  (3.1g)  into  (3. Id),  it  follows  that 

Finally,  applying  (3.1c)  again, 

E(z^)  -  ^h^^V^^     • 

If  2  is  such  that  E(z  )  does  not  vanish,  (z.)  has  a  linear  regression 
on  (J^q  y  2).  Remembering  (1.2a)  and  (1.2b),  the  foregoing  Indicates  a 

A  A 

Study  of  the  statistic   J^z^q^^y^^  ■  I^  y  2. 

Ij   ^J  J    i 

If  (1.2f)  holds,  it  can  be  shown  (Anscombe,  1961)  that 


A       A 


j  ij  ^j  "•  J 


10 


This  suggests  the  estimate  f  of  (t>  as  given  by  equation  (3.1a).  Then  If 

G-  ^ 


J/i/i^' 


and  W  >  (residual  SS  -  G) ,  G/W  Is  approximately  distributed  as  an  F  with 
one  and  (v-1)  degrees  of  freedom  and  Is  Tukey's  one  degree  of  freedom  test 
for  nonaddltlvlty  (Tukey,  19A9). 

For  the  two-way  classification  as  defined  In  Section  1.2, 


f  -  7 ' ^— 

^(SS-R)  (SS-C) 


^^'^  Kjyi.y.i 


f  - 


2(SS-R)(SS-C) 

If  removable  nonaddltlvlty  Is  found,  often 

P  -  1  -  2f  y 

will  Indicate  the  proper  power  transformation  (Tukey,  1957).  P  equal  to 
zero  Is  to  be  Interpreted  as  the  logarlthmetlc  transformation. 

3.2  Testing  Independence  of  the  Mean  and  Variance. 

In  order  to  measure  the  dependence  of  variability  upon  the  level  of 
measurement,  one  may  define  a  linear  regression  coefficient  of  the  (z.^) 
on  the  (y.).  Anscombe  (1961)  suggests 

(3.2a)  h-  ^^-:r- x -^     , 

I(qij)2(yi  -  Y)(yj  -  Y)S2   HS2 


11 


where  («. ) ,  (y.)  and  (q..)  are  as  defined  in  Section  1.2  and 


Assume  that  (y.  -  M.)  is  distributed  as  H(0,a^^),   where  o^^  ^s  propor- 
tional to  exp  (xy.).  Then  the  h  statistic  (3.2a)  is  an  estimate  of  x  • 
Motivation  for  the  h  statistic  comes  from  the  fact  that 


l\Hi^'   Y)]  -  I  (qij)2(Mi  -  M)(Mj  -  M)X02  +  OCx^) 


where 


h^<v 


ii"i 


and  where  O(x^)  is  the  regression  coefficient  of  z^^  on  (y^^  -  Y) ,  or 

e(z  2|y  -  y)  -  E(z  2)  +  O(x^).  See  Anscombe  (1961)  for  derivation  of  the 

h  statistic.  * 

If  one  desires  to  test  the  significance  of  the  deviation  of  x  f'O" 
zero  (equivalently,  whether  the  variance  is  dependent  on  the  level  of 
measurement) ,  a  mean  and  standard  error  for  the  conditional  distribution  of 
h,  given  the  fitted  values,  is  needed. 

Anscombe  (1961)  shows  that 

E(h|yJ  -  0   , 


and 


2v 


^"(^l^i)  ■  -UhWIa 


12 


For  a  one-way  classification, 


(k-l)SS (means) 
"-■^ k • 


where  k  is  the  number  of  observations  per  treatment. 
For  a  two-way  classification, 


^   ,  (r-2)(c-l)  (ss.j^j  ^  IlzlKczil  (ss-C)   . 


re  re 


For  an  r  x  r  latin  square  with  k  observations  per  cell 


H  -  <^"^) (^"^)  (SS-C  +  SS-R  +  SS-L)   . 

»2 


Anscombe  and  Tukey  (1963)  suggest  that  an  overestimate  of  H  can  be 
obtained  by  totaling  the  sum  of  squares  for  all  fitted  values  (treatments, 
blocks,  etc.) 

An  estimate  of  the  power  trfinsformation  necessary  for  making  the  error 
variance  constant   (x  ■  0)  is  given  by 

P  -  1  -  (l/2)h  Y  . 

For  the  two-way  classification,  Y  ■  y.   See  Tukey  (1957)  for  a  complete 
discussion  of  such  transformations. 

3.3  Normality. 

If  one  desires  to  test  hypotheses  using  the  results  from  a  least-squares 
analysis,  the  assumption  of  normality  (1.2i)  is  necessary  for  many 
procedures.   If  sufficiently  large  samples  are  taken,  most  sampling 
situations  produce  approximately  normal  distributions.  However,  the 


13 


consequences  of  grossly  non-normal  distributions  are  severe  enough  to  merit 

a  test  for  normality. 

In  assessing  the  shape  of  a  distribution,  the  skewnesa  coefficient  y^ 
and  the  kurtosis  coefficient  y   play  an  important  part.  They  are  defined 

2 


as 


-  3  . 


The  statistics  g   and  g  ,  both  functions  of  the  residuals,  will  be  proposed 

1         2 

as  estimates  of  Y   and  Y  •  They  are  due  to  Anscombe  (1961). 

12 

By  definition. 


i  ^      ^i  J  -•    ^ 


From  (3.1c), 


EClz,')  -  E{2(l,„(y,  -  .,))'} 


If  (1.2g)  and  (1.2h)  hold  and  since  the  errors  (y.  -  M.)  are  independent 
with  zero  means,  it  follows  that 


E(K'>  -  I<^ii>\'''  • 

i  1     ij   13    1 


It  is  clear  that 


W 


I  (<..,) 


3„3 


ij 


ij' 


is  a  possible  estimator  for  y  • 


14 


The  sampling  distribution  of  g   under  the  null  hypothesis  that 

Y  •■  0  is  determined  by 

I 

E(g  )  -  0,    Var  (g  )  -      ^ 


But,  g(g6)  .  (v-t-2)  (v+A)o6 

«2 


Using  (1.2e)  -  (1.21)  and  the  fact  that  2  ^b  p.s.d.,  Anscombe  (1961)  shows 
that 


Var(g  )  ^"' 


^    v(v+2)(v+4)(l  +  (n-l)p  3) 


where  p  ^  denotes  the  average  cubed  correlation  between  pairs  (z. ,z.). 
For  a  two-way  classification  (r  and  c>2) , 

°Kl'        ^   ,  ,           6nv 
g  -  J ,   Var  (g  )  -  . 

1   v(r-2)(c-2)s3  1    (\H-2)(v+4)(r-2)(c-2) 


Next,  consider  the  estimation  of  y   . 

2 

<\')    -  Eddq^.y^)-)   . 
1         1  J   -• 

From  (3.1c),  one  finds  that 


■;^,.-  , 


IS 


If  (1.2g)  and  (1.2h)  hold  and  since  the  errors  (y.  -  y.)  are  independent 
with  zero  means,  it  follows  that 


Edzi**)  -  I(qij'*)E(yj  -  Mj)'*  +  T   , 


where 


From  (3.1c)  and  since  ^  is  idempotent,  it  can  be  shown  that  (Anscombe,  1961) 

E(IO  -  I(qn)V''*-3^Si>'°''    • 

i  ^    ij  iJ   2     i  11 


Finally,  the  estimate  of  y   is 

2 


i       i 


V  +  2 


where 


Ij 


*ij' 


v(v  +  2) 


Under  the  null  hypothesis  that  y   "0  and  if  (1.2g)  and  (1.2h)  hold, 

2 

Anscombe  (1961)  showed  that 


E(g  )  -  0   , 
2 


and 


Var  (g  )  - 


24n2v2 


2    {v(v+2)(l  +  (n-l)p  •♦)-  3n}(\H-4)(v+6) 


16 


For  Che  two-way  classification  one  finds  that 

24n2v2 


Var  (g  ) 


2    {(\H-2)(r2-3r+3)(c2  -  3c+3)-3v2} (v+4) (v+6) 

It  is  of  interest  to  note  that  Srivastava  (1959)  has  found  that  skewness 
has  little  effect  on  the  power  of  the  F-test  in  a  one-way  analysis  of  variance, 
but  that  kurtosis  does. 

3. A  Reduction  to  Two-Way  Tables. 

Anscorabe  and  Tukey  (1963)  have  found  that,  "data  having  a  more  complex 
pattern  can  often  be  studied  effectively,  so  far  as  residuals  go,  in  a  two- 
way  classification."  This  is  the  motivation  for  listing  the  results  in  the 
previous  sections  for  a  two-way  classification. 

Data  in  a  2  factorial  experiment  Is  especially  amenable  to  residual 
analysis  based  on  a  two-way  structure.  Consider,  for  example,  a  2^  factorial 
experiment  in  one  replication.  If  one  assumes  that  very  few,  if  any,  of  the 
interactions  are  likely  to  be  important,  he  can  arrange  the  data  in  an 
8x8  table.  All  combinations  of  three  of  the  factors  will  constitute  the 
rows  while  the  columns  will  be  all  combinations  of  the  other  three  factors. 
Other  groupings  of  the  factors,  such  as  a  4  x  16  table,  are  also  possible. 
For  the  residual  analysis  to  be  effective,  the  following  two  things  are 
required: 

(1)  There  must  be  at  least  one  factor  in  each 
group  which  has  a  large  "effect". 

(2)  Any  extremely  large  interactions  must  be 
assigned  to  within-group,  rather  than 
between-group,  status. 
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4.   Modified  Residuals 

Most  of  the  procedures  presented  In  the  first  three  sections  of  this 
paper  are  sensitive  to  extreme  values  (outliers).   If  extreme  values  are 
present,  the  rejection  of  a  certain  hypothesis  may  be  due  solely  to  their 
presence,  rather  than  the  hypothesis  under  consideration  being  false.   Many 
papers  have  been  written  on  this  subject.   Only  one  procedure,  which  relies 
heavily  on  the  residuals,  will  be  presented  in  this  paper.   Also,  many  types 
of  complex  systematic  behavior  may  be  present  in  residuals  obtained  from  a 
conventional  analysis  which  fits  row,  column,  and  grand  means.   This  should 
be  taken  into  consideration  before  residuals  worthy  of  detailed  istudy  can  be 
obtained.   One  procedure  for  dealing  with  a  moderate  amount  of  this  complex 
systematic  behavior  is  the  vacuum  cleaner.   Since  the  vacuum  cleaner  is  also 
sensitive  to  extreme  values,  a  procedure  called  FUNOR-FUNOM  usually  precedes 
the  vacuum  cleaner. 

4.1  Modifying  Extreme  Values. 

Anscombe  and  Tukey  (1963)  propose  that  it  is  often  advantageous  to  have 
available  a  definite  rejection  rule  for  outliers.  This,  of  course,  will 
sometimes  lead  to  rejection  of  perfectly  good  observations.   Since  this  will 
tend  to  increase  the  average  error  variance,  one  can  regard  the  percentage 
increase  in  the  error  variance  as  the  premium  charged  for  protection  against 
bad  observations.  Realizing  that  bad  observations  lead  to  a  decrease  in 
precision,  one  can  see  that  this  is  a  reasonable  approach. 
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The  procedure  suggested  by  Anscombe  and  Tukey  (1963)  Is  to  reject  any 
observation  which  has  a  residual  greater  than  DS.  S  is  the  square  root 
of  the  error  mean  square  and,  If  a  •■  .05, 

D.  3.06(1  -i^)(^)l/2   ^ 

where  v  and  n  are  as  defined  In  Section  1.2. 

Many  authors  suggest  that  it  is  often  more  reasonable  to  modify  an 
observation  than  to  reject  it  completely  {see  Jeffreys  (1948) ,  de  Finetti 
(1961),  and  Anscombe  and  Tukey  (1963)}.  Tukey  (1962)  presents  a  procedure 
called  FUNOR-FUNOM  which  accomplishes  the  necessary  modification. 

A  less  complicated  procedure,  first  suggested  by  C.  P.  Winsor  (see 
Dixon,  1960),  is  to  replace  any  suspected  observation  by  the  nearest  non- 
suspect  observation.  Therefore,  any  observation  containing  a  residual  which 
exceeds  DS  by  only  a  small  amount  can  be  reduced  by  the  difference  in  its 
residual  and  the  largest  nonsuspect  residual.  Any  observation  that  has  a 
residual  exceeding  DS  '  by  a  large  amount  will  be  rejected  completely  unless 
there  is  definite  reason  for  believing  this  observation  is  correct.  If  an 
.observation  is  rejected  or  modified,  the  standard  error  S  should  be 
recomputed  before  testing  another  potential  extreme  value. 

A. 2  The  Vacuum  Cleaner. 

The  vacuum  cleaner  is  a  procedure  presented  by  Tukey  (1962)  which  yields 
residuals  freer  of  systematic  structures  than  the  conventional  mean  fitting. 
Since  some  aberrations  often  simulate  others,  it  is  occasionally  desirable 
to  apply  this  procedure  before  the  procedures  In  Section  3  are  applied.  It 
should  be  noted,  however,  that  if  this  is  done,  the  procedures  in  Section  3 
will  have  to  be  modified. 
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Tukey  (1962)  first  constructed  a  subprocedure  which  regressed  the  values 
in  each  row  of  a  two-way  table  on  the  values  in  a  separately  given  row,  next 
regressed  the  values  in  each  column  on  the  values  in  a  separately  given 
column,  thirdly,  regressed  the  whole  table  on  the  two-way  array  consisting  of 
all  products  of  an  entry  in  the  separate  row  with  an  entry  in  the  separately 
given  column,  and  finally,  subtracts  this  last  regression  from  each  of  the   -  ; 
other  two.  The  result  is  a  four-part  breakdown: 

(original  values)  -  (dual  regression)  +  (deviations  of  row 

regression  from  dual  regression)  + 
(deviations  of  column  regression  from 
dual  regression)  +  (residuals). 

The  basic  vacuum  cleaner  becomes  the  application  of  this  subprocedure  twice. 
The  first  application  removes  row,  column,  and  grand  means.  The  second 
application  removes  row-by-row  regression  upon  "column  mean  minus  grand  mean" 
and  column-by-column  regression  on  "row  mean  minus  grand  mean".  The  dual 
regression  of  the  second  application  is  the  one  degree  of  freedom  for  non- 
additivity. 

5.   An  Example 

The  following  example  (Table  5. a)  presented  by  Gill  (1966)  will  serve 
to  Illustrate  some  of  the  techniques  presented  in  the  previous  sections. 
Results  of  the  conventional  analysis  are  given  in  Table  5.b  and  the  resulting  ; 
residuals  in  Table  5.c. 
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Table  3. a 


Oil  Content 

of  Flaxseed  Inoculated  at  Different  Stages 

of  Growth 

Treatment 

Block 

1 

2 

3 

A 

^i. 

h. 

Seedling 

4. A 

5.9 

6.0 

A.l 

20. A 

5.1 

Early  Bloom 

3.3 

1.9 

A. 9 

7.1 

17.2 

A. 3 

Full  Bloom 

A. 4 

A.O 

A. 5 

3.1 

16.0 

A.O 

Full  (1/100) 

6.8 

6.6 

7.0 

6. A 

26.8 

6.7 

Ripening 

6.3 

A. 9 

5.9 

7.1 

2A.2 

6.05 

Uninoculated 

6. A 

7.3 

7.7 

6.7 

28.1 

7.02 

".J 

31.6 

30.6 

36.0 

3A.5 

132.7 

''■i 

5.27 

5.1 

6.0 

5.75 

y-5.53 

Table  5.b 


Analysis  of  Variance 

Source 

dF 

SS 

MS 

F-test 

Blocks 

Treatments 

Residual 

3 

5 

15 

3.1A 
31.65 
19.72 

1.05 
6.33 
1.31 

A. 83     p(<.01) 

Total 

23 

5A.51 

Table  5.c 
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Residuals 

J 

^i1 

^i1 

^i 

rank-k 

\/n 

<'n-^>/\/n 

1 

4.4 

4.84 

-0.44 

16 

-0.37 

1.40 

2 

5.9 

4.67 

+1.23 

2 

+1.50 

0.77 

3 

6.0 

5.57 

+0.43 

7  tie 

40.54* 

0.65 

4 

4.1 

5.32 

-1.22 

23 

-1.50 

0.87 

2, 

1 

3.3 

4.04 

-0.74 

21 

-1.04 

0.79 

2, 

2 

1.9 

3.87 

-1.97 

24 

-1.95 

1.05 

2, 

3 

4.9 

4.77 

40.13 

12 

+0.05 

1.00 

2, 

4 

7.1 

4.52 

+2.58 

1 

+1.95 

1.28 

3, 

1 

4.4 

3.74 

+0.66 

5 

+0.88 

0.66 

3, 

2 

4.0 

3.57 

+0.43 

7  tie 

+0.54* 

0.65 

3, 

3 

4.5 

4.47 

+0.03 

13 

-0.05 

1.00 

3 

4 

3.1 

4.22 

-1.12 

22 

-1.24 

0.97 

4 

,   1 

6.8 

6.44 

+0.36 

9 

+0.37 

0.76 

4 

.   2 

6.6 

6.27 

+0.33 

10 

+0.26 

0.96 

4 

,   3 

7.0 

7.17 

-0.17 

14 

-0.16 

1.56 

4 

,   4 

6.4 

6.92 

-0.52 

17 

-0.48 

1.25 

5 

,   1 

6.3 

5.79 

+0.51 

6 

+0.73 

0.59 

5 

,   2 

4.9 

5.62 

-0.72 

20 

-0.88 

0.91 

5 

,   3 

5.9 

6.52 

-0.62 

19 

-0.73 

0.96 

5 

,   4 

7.1 

6.27 

40.83 

3 

+1.24 

0.60 

6 

,   1 

6.4 

6.76 

-0.36 

15 

-0.26 

1.69 

6 

,   2 

7.3 

6.59 

+0.71 

4 

+1.04 

0.60 

6 

»   3 

7.7 

7.49 

40.21 

11 

+0.16 

0.81 

6 

,  4 

6.7 

7.24 

-0.54 

18 

-0.60 

1.03 

y-5.53 

z- (+0.13+0. 03) /2 
-+0.08 

*ave.   0 
(+0.604 

f  7  &  8 

0. 48)72-0.54 
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The  first  step  is  to  plot  the  residuals  against  the  fitted  values  (see 
Section  2.1).  Figure  5.d  gives  this  plot. 


Figure-  5id 
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The  following  results  are  noted: 


.'   «l....5  *' 


(1)  Two  "potential"  outliers  are  Early  Bloom, 
Blocks  2  and  4. 

(2)  Excluding  these  two  values,  the  variance 

.  apparently  is  independent  of  the  level  of 
response. 

(3)  The  assumption  of  additivity  (1.2g)  appears 
to  be  met  since  the  plot  is  linear. 
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The  second  step  is  FUNOP  (2.2).  This  result  is  given  in  Figure  5.e. 


Figure  5.e 
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The  following  results  are  noted: 

'  •       (1)  One  "potential"  outlier  is  Early  Bloom,  Block  4. 

(2)  The  upturned  tails  give  sope  indication  of  "•'',>'■{ 
kurtosis;  the  higher  plot  on  the  right       \  j 
indicates  some  skewness.  This  causes  one 
to  believe  that  the  assumption  of  normality 
may  not  be  met. 
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The  next  step  is  to  apply  the  procedure  suggested  in  Section  4.1.  The 
following  calculations  are  noted: 


(1)  S  -  (1.31) 


1/2 


1.145 


(2)  D  -  3.06  (1  -^  )(v/n)^^^ 


-  3.06  (1  - 


-  2.12 


V 

1.85 
15 


)(15/24) 


1/2 


(3)  DS  -  2.43 

Since  the  residual  for  Early  Bloom  in  Block  4  (2.58)  exceeds  DS  by  a 
small  amount,  it  will  be  modified.  The  nearest  nonsuspect  residual  is. 1.23. 
The  modified  value  is  then  7.1  -  (2.58-1.23)  -  5.75.  A  second  two-way 
analysis,  using  the  modified  value,  yields  Table  5.f. 

Table  5.f 


Analysis  of  Variance  after  Modification  of  First  Outlier 

Source 

dF 

SS 

MS 

F-test 

Blocks 

Treatments 

Residual 

3 

5 

15 

2.77 

35.35 

3.89 

0.92 
7.07 
0.26 

27.19     p(<.01) 

Total 

23 

42.01 

This  result  is  quite  startling.   By  simply  modifying  one  observation, 
the  residual  mean  square  has  been  reduced  by  80%  of  its  original  value.  When 
the  sole  purpose  of  an  experiment  is  to  test  for  treatment  differences,  the 
validity  of  using  such  a  procedure  on  a  small  data  set  is  questionable. 


vr.^'  ^ 
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However,  for  the  purpose  of  testing  the  assumptions  necessary  for  doing  an 
analysis  of  variance,  this  procedure  seems  to  be  adequate.  By  repeating  the 
method  of  Section  A.l  the  other  suspected  outlier  (Early  Bloom,  Block  2)  is 
also  modified  (from  li9  to  2.6A).  No  other  observations  are  modified  as  the 
procedure  is  repeated.  ^  • 

The  next  step  is  the  second  scatter  plot  (2.3).  Figure  5.g  gives  the  ■ 
desired  plot.  The  obvious  lack  of  a  linear  trend  Indicates  that  the 
assumption  of  additivlty  is  probably  satisfied.         .        \ 
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Figure  5.g 
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Next,  consider  the  test  statistics  presented  in  Section  3.  The  analysis 
of  variance  after  modification  of  both  outliers  is  given  in  Table  5.h. 


Table  5.h 


Analysis  of  Variance  after  Modification  of  Both  Outliers 

Source 

dF 

SS 

MS 

F-test 

Blocks 

Treatments 

Residual 

3 

5 
15 

2.29 

A1.77 

3.19 

0.76 
8.35 
0.21 

39.76     p(<.01) 

Total 

23 

47.25 

To  test  for  removable  non-addltivity,  consider  the  following: 


f  -  T  - 


Kjyj.y. 


1 


-1.43 


A   2(SS-R)(SS-C)   12.02   * 


where  the  z . .  are  the  residuals  after  modification  of  the  suspect  observations. 
Also,  consider  the  following: 


J-.0.17  . 


and  the  remainder  sum  of  squares  is  (3.19  -  0.17)  -  3.02.  Consequently, 
F(l,14)  -  0.17/3.02  -  0.06,  so  it  appears  that  the  assumption  of  additivity 
is  met. 
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To  test  for  dependence  of  variability  upon  the  level  of  response, 
consider 

H  -  ((r-2)(c-l)(SS-R)/rc)  +  ((r-1) (c-2) (SS-C)/rc) 

-  ((4)(3)(A1.77)/24)  +  ((5) (2) (2.29)/2A) 

-  21.84 

-  -6.335/(3.19) (21.84) 

-  -0.09 

Var(h)  -  2v/((v+2)h) 

-  2(15)/((17)(21.84)) 

-  0.08 


To  test  H  :  X  ■  0. 
o  '^ 


t(15dF)  -  h/(Var(h))^^^  -  -0.32 


There  is  no  good  evidence  for  dependence  of  variability  on  the  level  of 
response. 

To  check  on  the  assumption  of  normality,  consider 

8  -  nj;z^j3/(v(r-2)(c-2)s3) 

-  (20)(-3.034)/((15)(4)(2)(3.19)^/^) 

-  -0.089 

Var(g  )  -  6nv/((v+2)(v+4)(r-2)(c-2)) 
1       ^  ' 

-  (6)  (20)  (15)/  ((15+2)  (15+4)  (6-2)  (4-2)) 

-  0.697 


n2v^{[(^)():z^/)/(vs2)2]  -  3/n} 


(v+2)  (r2-3r+3)  (c2-3c+3)-3v2 


(20)2(15)2. 


,15+2 


(^)(13.67)/(15(3.19)]2 


15 


-  3/20 


(15+2) (62-18+3) (42-12+3)-3(15)2 


7.07 


Var(g  )  -  24n2v2/{((v+2)(r2-3irf3)(c2-3c+3)-3v2)(v+4)(v+6)} 

-  24(20)2(15)2/(1824) (19) (21) 

-  2.97 


To  test  H  :  Y  "0  (no  skewness  present), 


t(15dF)  -  g  /(Var(g  ))^^^-  -0.107  . 


There  is  no  evidence  of  skewness. 


To  test  H  :  Y  *  0  (no  kurtosis  present) ,  use 
o   2 


t(15dF)  -  g  /(Var(g  ))^^^-  4.11  . 
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There  may  be  kurtosis  present.  This  would  need  confirmation  with  other  data 
sets.   Since  kurtosis  is  a  flattening  out  of  a  distribution,  its  presence 
would  make  true  treatment  differences  more  difficult  to  detect. 

6.   Conclusion 


The  techniques  in  this  paper  provide  useful  insight  into  the  validity 
of  the  assumptions  in  a  classical  least-squares  analysis.   They  are  fairly 
easy  to  apply,  but  have  no  sweeping  statistical  properties.  Not  all  possible 
information  about  these  assumptions  can  be  obtained  from  the  residuals  and 
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the  results  obtained  must  be  qualified  in  the  sense  that  it  usually  requires 
several  data  sets  or  large  amounts  of  data  to  determine  such  things  as  the 
shape  of  the  distribution  with  any  degree  of  confidence.   One  should  also 
remember  that  most  of  these  techniques  are  sensitive  to  outliers  and  that  it 
is  very  easy  for  one  violation  to  act  like  another.   One  should  allow  these 
techniques  to  guide  his  judgment  but  not  dictate  courses  of  action.   If, 
however,  gross  violations  of  these  assumptions  are  detected,  one  should 
consider  either  using  nonparametric  techniques  (see  Lohrding,  1966)  or 
making  a  transformation  of  the  data. 
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ABSTRACT 

This  paper  provides  methods  of  testing  the  validity  of  the  assumptions 
necessary  for  using  the  classical  least-squares  procedures.   These  procedures 
are  based  on  the  residuals — the  observed  value  minus  the  fitted  value.   Both 
graphic  techniques  and  test  statistics  are  presented  for  detecting  removable 
nonaddltlvlty,  dependence  of  variance  on  the  level  of  measurement,  and  non- 
normality. 

The  first  graphic  technique  Is  a  simple  plot  of  the  residuals  against 
the  fitted  values.   It  Is  the  first  step  In  gaining  Insight  Into  the  appro- 
priateness of  the  least-squares  techniques.   The  second  graphic  technique 
presented,  called  FUNOP,  uses  the  residuals  In  a  somewhat  more  complicated 
way  to  Indicate  the  shape  of  the  underlying  distribution.   A  third  graphic 
technique,  called  the  second  scatter  plot.  Is  presented  for  detecting 
removable  nonaddltlvlty. 

The  test  statistics  are  presented  for  detecting  removable  nonaddltlvlty, 
dependence  of  variance  on  the  level  of  measurement,  skewness,  and  kurtosis 
(f,  h,  g  ,  and  g  ,  respectively).  The  mean  and  variance  for  each  statistic 
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is  found  under  the  null  hypothesis  that  the  corresponding  population  parameter 
which  the  statistic  estimates  Is  zero. 

The  above  procedures  are  presented  only  as  a  guide  to  better  data 
analysis,  not  as  completely  valid  techniques  with  sweeping  optimum  properties. 
If  applied  along  with  good  statistical  judgment,  they  are  excellent  guides 
for  handling  important  problems  faced  in  data  analysis. 


